{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 百度AI——驾驶行为分析\n",
    "针对车载场景，识别驾驶员使用手机、抽烟、不系安全带、双手离开方向盘等动作姿态，分析预警危险驾驶行为，提升行车安全性。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 开始我们的实验\n",
    "(1)登录百度AI开发平台，选择“人体分析”，使用“创建应用”建立一个驾驶行为分析应用。 获取应用的AppID、API Key、Secret Key，注意:Secret Key要先选择显示，然后复制其现实的字符。\n",
    "\n",
    "例如我创建的驾驶行为分析应用获取的AppID、API Key、Secret Key如下：\n",
    "\n",
    "    \"\"\" 公开课的限定量人体行为分析，需要更改为自己的api接口 \"\"\"\n",
    "    pic_APP_ID = '21258767'\n",
    "    pic_API_KEY = 'epRpthFLqIW7PHNWfYP9NqK5'\n",
    "    pic_SECRET_KEY = 'RF2yO8OIfvcrI4WlC3w0In0G6Upo7GFK'\n",
    "\n",
    "    \n",
    "百度AI开发平台注册链接:https://login.bce.baidu.com/?account=&redirect=http%3A%2F%2Fconsole.bce.baidu.com%2F%3Ffromai%3D1#/aip/overview\n",
    "\n",
    "(2)熟悉我封装的百度A驾驶行为分析的模块 在代码中输入如下代码，导入我的封装的模块\n",
    "\n",
    "    import sys\n",
    "    sys.path.append('../baidu_api_lib')\n",
    "    from baidu_picture import baidu_picture_2_msg\n",
    "    封装库提供两个方法：初始化方法，驾驶行为分析方法。\n",
    "\n",
    "<1>初始化方法，需要传入三个参数，就是上步我们创建“驾驶行为分析”应用获取的三个参数 参数信息如下：\n",
    "\n",
    "    \"\"\" 公开课的限定量人体行为分析，需要更改为自己的api接口 \"\"\"\n",
    "    pic_APP_ID = '21258767'\n",
    "    pic_API_KEY = 'epRpthFLqIW7PHNWfYP9NqK5'\n",
    "    pic_SECRET_KEY = 'RF2yO8OIfvcrI4WlC3w0In0G6Upo7GFK'\n",
    "\n",
    "调用方式：\n",
    "\n",
    "    # 传入百度AI的参数，进行图像识别\n",
    "    pic_msg = baidu_picture_2_msg(pic_APP_ID, pic_API_KEY, pic_SECRET_KEY)\n",
    "    初始化完毕后，获取pic_msg对象，操作该对象就能进行动物识别。\n",
    "<2>驾驶行为分析方法 这是我封装百度驾驶行为分析方法的函数原型：\n",
    "\n",
    "    # 传入百度AI的参数，进行图像识别\n",
    "    pic_msg = baidu_picture_2_msg(pic_APP_ID, pic_API_KEY, pic_SECRET_KEY)\n",
    "    \n",
    "    baidu_request_url:百度AI的调用接口，\n",
    "    #百度AI的调用url\n",
    "    baidu_request_url = \"https://aip.baidubce.com/rest/2.0/image-classify/v1/animal\"\n",
    "\n",
    "    'read_word.jpg':进行驾驶行为分析的图片。\n",
    "    返回值:response为文字识别的信息\n",
    "\n",
    "    response.json()[\"person_num\"]:检测到的总人数（包括驾驶员和乘客），0代表未监测到驾驶员\n",
    "    response.json()[\"driver_num\"]:检测到的驾驶员数目。\n",
    "    response.json()[\"person_info\"]:驾驶员的属性行为信息；若未检测到驾驶员，则该项为[]\n",
    "    response.json()[\"person_info\"][0][attributes]:第一个驾驶员的所有行为分析结果，包含是否吸烟、是否使用手机、是否未系安全带等。\n",
    "\n",
    "    ...还有不少参数，通过print(response.json())可以发现"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "######################################################\n",
    "#\n",
    "# 先感受下小派驾驶行为分析的本领,选择本cell，按shirt+enter键运行本模块\n",
    "#\n",
    "######################################################\n",
    "\n",
    "#!/usr/bin/env python\n",
    "# -*- coding: utf-8 -*-\n",
    "#\n",
    "# Copyright (c) 2020 Taste all Pi.\n",
    "#\n",
    "# Licensed under the GNU General Public License, Version 2.0 (the \"License\");\n",
    "# you may not use this file except in compliance with the License.\n",
    "# You may obtain a copy of the License at\n",
    "#\n",
    "#   http://www.gnu.org/licenses/gpl-2.0.html\n",
    "#\n",
    "# Unless required by applicable law or agreed to in writing, software\n",
    "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
    "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
    "# See the License for the specific language governing permissions and\n",
    "# limitations under the License.\n",
    "\n",
    "#导入标准库\n",
    "import sys\n",
    "import os\n",
    "from playsound import playsound\n",
    "import cv2 as cv\n",
    "import time\n",
    "from multiprocessing import Process, Queue\n",
    "import multiprocessing\n",
    "\n",
    "#导入自定义库\n",
    "sys.path.append('../baidu_api_lib')\n",
    "from baidu_picture import baidu_picture_2_msg\n",
    "from baidu_sound import baidu_word_2_sound\n",
    "\n",
    "\"\"\" 公开课的限定量人体行为分析，需要更改为自己的api接口 \"\"\"\n",
    "pic_APP_ID = '21258767'\n",
    "pic_API_KEY = 'epRpthFLqIW7PHNWfYP9NqK5'\n",
    "pic_SECRET_KEY = 'RF2yO8OIfvcrI4WlC3w0In0G6Upo7GFK'\n",
    "\n",
    "#百度AI的调用url\n",
    "baidu_request_url = \"https://aip.baidubce.com/rest/2.0/image-classify/v1/driver_behavior\"\n",
    "\n",
    "#　获取摄像头图像\n",
    "def camera_frame_func(task_name, mult_queue1, mydict):\n",
    "    \n",
    "    # 创建一个VideoCapture对象\n",
    "    capture = cv.VideoCapture(0) \n",
    "    \n",
    "    #更新人脸识别信息\n",
    "    mydict[\"get_pic\"] = \"false\"\n",
    "    \n",
    "    #　给出提示信息\n",
    "    print(task_name + \"任务启动\")\n",
    "    \n",
    "    try:\n",
    "        while True:\n",
    "            # 一帧一帧读取视频\n",
    "            ret, frame = capture.read()\n",
    "            \n",
    "            #将拍摄到的图片发送到消息队列中\n",
    "            if mydict[\"get_pic\"] == \"true\":\n",
    "                mydict[\"get_pic\"] = \"false\"\n",
    "                mult_queue1.put(frame)\n",
    "\n",
    "            # 本地显示视频图像\n",
    "            cv.imshow('real_time picture', frame) \n",
    "            cv.waitKey(1)\n",
    "\n",
    "    except KeyboardInterrupt:\n",
    "        # 释放cap,销毁窗口\n",
    "        capture.release()    \n",
    "        #关闭显示窗口\n",
    "        cv.destroyAllWindows() \n",
    "        print(task_name + \"任务被终止\")\n",
    "        \n",
    "#处理图像\n",
    "def proc_frame_func(task_name, mult_queue, mydict):\n",
    "    #　给出提示信息\n",
    "    print(task_name + \"任务启动\")\n",
    "    \n",
    "    # 传入百度AI的参数，进行图像识别\n",
    "    pic_msg = baidu_picture_2_msg(pic_APP_ID, pic_API_KEY, pic_SECRET_KEY)\n",
    "    \n",
    "    #传入百度AI的参数，进行语音合成\n",
    "    word_2_sound = baidu_word_2_sound(pic_APP_ID, pic_API_KEY, pic_SECRET_KEY)\n",
    "    word_2_sound.trans_word_to_sound(\"欢迎使用小派安全驾驶辅助系统\",'tst_sound.mp3')\n",
    "    os.system('mplayer ' + 'tst_sound.mp3')\n",
    "    time.sleep(2)\n",
    "    \n",
    "    try:\n",
    "       \n",
    "        while True:\n",
    "            #连续获取图片，取出第三章\n",
    "            loop_num = 0\n",
    "            while loop_num < 3:\n",
    "                #从队列中获取图片，显示图像质量\n",
    "                mydict[\"get_pic\"] = \"true\"\n",
    "                frame = mult_queue.get()\n",
    "                cv.waitKey(1)\n",
    "                loop_num += 1\n",
    "                time.sleep(1)\n",
    "            \n",
    "            #关闭显示窗口\n",
    "            cv.destroyAllWindows()            \n",
    "\n",
    "            # 写入图片\n",
    "            cv.imwrite('read_word.jpg',frame)\n",
    "            \n",
    "            #从百度AI获取图片分析结果\n",
    "            response = pic_msg.pic_2_msg(baidu_request_url, 'read_word.jpg')\n",
    "            \n",
    "            #给出百度AI分析的数据\n",
    "            print(response.json())\n",
    "            \n",
    "            #识别出驾驶员了，就要根据识别信息给出提示\n",
    "            if response.json()[\"driver_num\"] > 0:\n",
    "                #获取驾驶员的驾驶状态\n",
    "                driver_attr = response.json()['person_info'][0]['attributes']\n",
    "                \n",
    "                #未佩戴安全带概率大于0.5（概率大于50%）\n",
    "                if driver_attr['not_buckling_up']['score'] > 0.5:\n",
    "                    word_2_sound.trans_word_to_sound(\"安全驾驶，请先系好安全带\",'tst_sound.mp3')\n",
    "                    os.system('mplayer ' + 'tst_sound.mp3')\n",
    "                    \n",
    "                #未带口罩概率大于0.5（概率大于50%）\n",
    "                if driver_attr['no_face_mask']['score'] > 0.5:\n",
    "                    word_2_sound.trans_word_to_sound(\"请带好口罩，为抗疫胜利尽份力\",'tst_sound.mp3')\n",
    "                    os.system('mplayer ' + 'tst_sound.mp3')\n",
    "                     \n",
    "                #打电话概率大于0.5（概率大于50%）\n",
    "                if driver_attr['cellphone']['score'] > 0.5:\n",
    "                    word_2_sound.trans_word_to_sound(\"安全驾驶，请不要拨打电话\",'tst_sound.mp3')\n",
    "                    os.system('mplayer ' + 'tst_sound.mp3')\n",
    "                    \n",
    "                #双手远离方向盘概率大于0.5（概率大于50%）\n",
    "                elif driver_attr['both_hands_leaving_wheel']['score'] > 0.5:\n",
    "                    word_2_sound.trans_word_to_sound(\"安全驾驶，请握紧方向盘\",'tst_sound.mp3')\n",
    "                    os.system('mplayer ' + 'tst_sound.mp3')    \n",
    "                       \n",
    "                #打哈欠概率大于0.5（概率大于50%）\n",
    "                elif driver_attr['yawning']['score'] > 0.5:\n",
    "                    word_2_sound.trans_word_to_sound(\"您有点困了，请停车休息\",'tst_sound.mp3')\n",
    "                    os.system('mplayer ' + 'tst_sound.mp3')      \n",
    "                    \n",
    "                #低头概率大于0.5（概率大于50%）\n",
    "                elif driver_attr['head_lowered']['score'] > 0.5:\n",
    "                    word_2_sound.trans_word_to_sound(\"安全驾驶，请注视前方\",'tst_sound.mp3')\n",
    "                    os.system('mplayer ' + 'tst_sound.mp3')      \n",
    "                    \n",
    "                #吸烟概率大于0.5（概率大于50%）\n",
    "                elif driver_attr['smoke']['score'] > 0.5:\n",
    "                    word_2_sound.trans_word_to_sound(\"为了您和他人的健康，请不要在车内吸烟\",'tst_sound.mp3')\n",
    "                    os.system('mplayer ' + 'tst_sound.mp3')\n",
    "                \n",
    "                #其他情况，可以自行添加\n",
    "                else:\n",
    "                    pass\n",
    "            \n",
    "            #等待10秒后，在进行图像采集\n",
    "            time.sleep(10)\n",
    "            \n",
    "    except KeyboardInterrupt:\n",
    "        os.remove('tst_sound.mp3')\n",
    "        os.remove('read_word.jpg')\n",
    "        print(task_name + \"任务被终止\")\n",
    "        \n",
    "if __name__ == \"__main__\":\n",
    "    try:\n",
    "        \n",
    "        mydict=multiprocessing.Manager().dict()\n",
    "        \n",
    "        #　定义传递图像队列和传递图像处理结果队列\n",
    "        q_frame = Queue()\n",
    "        \n",
    "        #　采集摄像头进程、处理图片进程、播报语音信息\n",
    "        get_camera_frame = Process(target=camera_frame_func, args=(\"获取摄像头图像\", q_frame, mydict))\n",
    "        proc_frame       = Process(target=proc_frame_func, args=(\"处理图像\", q_frame, mydict))\n",
    "        \n",
    "        # 启动任务\n",
    "        get_camera_frame.start()\n",
    "        proc_frame.start()\n",
    "\n",
    "    except KeyboardInterrupt:\n",
    "        print(\"任务被终止了\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 让我们临摹代码，学习“驾驶行为分析”功能"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# #导入标准库\n",
    "# import sys\n",
    "\n",
    "# import os\n",
    "\n",
    "# from playsound import playsound\n",
    "\n",
    "# import cv2 as cv\n",
    "\n",
    "# import time\n",
    "\n",
    "# from multiprocessing import Process, Queue\n",
    "\n",
    "# import multiprocessing\n",
    "\n",
    "\n",
    "# #导入自定义库\n",
    "# sys.path.append('../baidu_api_lib')\n",
    "\n",
    "# from baidu_picture import baidu_picture_2_msg\n",
    "\n",
    "# from baidu_sound import baidu_word_2_sound\n",
    "\n",
    "\n",
    "# \"\"\" 公开课的限定量人体行为分析，需要更改为自己的api接口 \"\"\"\n",
    "# pic_APP_ID = '21258767'\n",
    "\n",
    "# pic_API_KEY = 'epRpthFLqIW7PHNWfYP9NqK5'\n",
    "\n",
    "# pic_SECRET_KEY = 'RF2yO8OIfvcrI4WlC3w0In0G6Upo7GFK'\n",
    "\n",
    "\n",
    "\n",
    "# #百度AI的调用url\n",
    "# baidu_request_url = \"https://aip.baidubce.com/rest/2.0/image-classify/v1/driver_behavior\"\n",
    "\n",
    "\n",
    "# #　获取摄像头图像\n",
    "# def camera_frame_func(task_name, mult_queue1, mydict):\n",
    "\n",
    "    \n",
    "#     # 创建一个VideoCapture对象\n",
    "#     capture = cv.VideoCapture(0) \n",
    "\n",
    "    \n",
    "#     #更新人脸识别信息\n",
    "#     mydict[\"get_pic\"] = \"false\"\n",
    "\n",
    "    \n",
    "#     #　给出提示信息\n",
    "#     print(task_name + \"任务启动\")\n",
    "\n",
    "    \n",
    "#     try:\n",
    "\n",
    "#         while True:\n",
    "\n",
    "#             # 一帧一帧读取视频\n",
    "#             ret, frame = capture.read()\n",
    "\n",
    "            \n",
    "#             #将拍摄到的图片发送到消息队列中\n",
    "#             if mydict[\"get_pic\"] == \"true\":\n",
    "\n",
    "#                 mydict[\"get_pic\"] = \"false\"\n",
    "\n",
    "#                 mult_queue1.put(frame)\n",
    "\n",
    "\n",
    "#             # 本地显示视频图像\n",
    "#             cv.imshow('real_time picture', frame) \n",
    "\n",
    "#             cv.waitKey(1)\n",
    "\n",
    "\n",
    "#     except KeyboardInterrupt:\n",
    "\n",
    "#         # 释放cap,销毁窗口\n",
    "#         capture.release()   \n",
    "\n",
    "#         #关闭显示窗口\n",
    "#         cv.destroyAllWindows() \n",
    "\n",
    "#         print(task_name + \"任务被终止\")\n",
    "\n",
    "        \n",
    "# #处理图像\n",
    "# def proc_frame_func(task_name, mult_queue, mydict):\n",
    "\n",
    "#     #　给出提示信息\n",
    "#     print(task_name + \"任务启动\")\n",
    "\n",
    "    \n",
    "#     # 传入百度AI的参数，进行图像识别\n",
    "#     pic_msg = baidu_picture_2_msg(pic_APP_ID, pic_API_KEY, pic_SECRET_KEY)\n",
    "\n",
    "    \n",
    "#     #传入百度AI的参数，进行语音合成\n",
    "#     word_2_sound = baidu_word_2_sound(pic_APP_ID, pic_API_KEY, pic_SECRET_KEY)\n",
    "\n",
    "#     word_2_sound.trans_word_to_sound(\"欢迎使用小派安全驾驶辅助系统\",'tst_sound.mp3')\n",
    "\n",
    "#     os.system('mplayer ' + 'tst_sound.mp3')\n",
    "\n",
    "#     time.sleep(2)\n",
    "\n",
    "    \n",
    "#     try:\n",
    "\n",
    "       \n",
    "#         while True:\n",
    "\n",
    "#             #连续获取图片，取出第三章\n",
    "#             loop_num = 0\n",
    "\n",
    "#             while loop_num < 3:\n",
    "\n",
    "#                 #从队列中获取图片，显示图像质量\n",
    "#                 mydict[\"get_pic\"] = \"true\"\n",
    "\n",
    "#                 frame = mult_queue.get()\n",
    "\n",
    "#                 cv.imshow('I get the picture', frame) \n",
    "\n",
    "#                 cv.waitKey(1)\n",
    "\n",
    "#                 loop_num += 1\n",
    "\n",
    "#                 time.sleep(1)\n",
    "\n",
    "            \n",
    "#             #关闭显示窗口\n",
    "#             cv.destroyAllWindows()    \n",
    "\n",
    "\n",
    "#             # 写入图片\n",
    "#             cv.imwrite('read_word.jpg',frame)\n",
    "\n",
    "            \n",
    "#             #从百度AI获取图片分析结果\n",
    "#             response = pic_msg.pic_2_msg(baidu_request_url, 'read_word.jpg')\n",
    "\n",
    "            \n",
    "#             #给出百度AI分析的数据\n",
    "#             #print(response.json())\n",
    "\n",
    "            \n",
    "#             #识别出驾驶员了，就要根据识别信息给出提示\n",
    "#             if response.json()[\"driver_num\"] > 0:\n",
    "\n",
    "#                 #获取驾驶员的驾驶状态\n",
    "#                 driver_attr = response.json()['person_info'][0]['attributes']\n",
    "\n",
    "                \n",
    "#                 #未佩戴安全带概率大于0.5（概率大于50%）\n",
    "#                 if driver_attr['not_buckling_up']['score'] > 0.5:\n",
    "\n",
    "#                     word_2_sound.trans_word_to_sound(\"安全驾驶，请先系好安全带\",'tst_sound.mp3')\n",
    "\n",
    "#                     os.system('mplayer ' + 'tst_sound.mp3')\n",
    "\n",
    "                    \n",
    "#                 #未带口罩概率大于0.5（概率大于50%）\n",
    "#                 if driver_attr['no_face_mask']['score'] > 0.5:\n",
    "\n",
    "#                     word_2_sound.trans_word_to_sound(\"请带好口罩，为抗疫胜利尽份力\",'tst_sound.mp3')\n",
    "\n",
    "#                     os.system('mplayer ' + 'tst_sound.mp3')\n",
    "\n",
    "                     \n",
    "#                 #打电话概率大于0.5（概率大于50%）\n",
    "#                 if driver_attr['cellphone']['score'] > 0.5:\n",
    "\n",
    "#                     word_2_sound.trans_word_to_sound(\"安全驾驶，请不要拨打电话\",'tst_sound.mp3')\n",
    "\n",
    "#                     os.system('mplayer ' + 'tst_sound.mp3')\n",
    "\n",
    "                    \n",
    "#                 #双手远离方向盘概率大于0.5（概率大于50%）\n",
    "#                 elif driver_attr['both_hands_leaving_wheel']['score'] > 0.5:\n",
    "\n",
    "#                     word_2_sound.trans_word_to_sound(\"安全驾驶，请握紧方向盘\",'tst_sound.mp3')\n",
    "\n",
    "#                     os.system('mplayer ' + 'tst_sound.mp3')    \n",
    "\n",
    "                       \n",
    "#                 #打哈欠概率大于0.5（概率大于50%）\n",
    "#                 elif driver_attr['yawning']['score'] > 0.5:\n",
    "\n",
    "#                     word_2_sound.trans_word_to_sound(\"您有点困了，请停车休息\",'tst_sound.mp3')\n",
    "\n",
    "#                     os.system('mplayer ' + 'tst_sound.mp3')      \n",
    "\n",
    "                    \n",
    "#                 #低头概率大于0.5（概率大于50%）\n",
    "#                 elif driver_attr['head_lowered']['score'] > 0.5:\n",
    "\n",
    "#                     word_2_sound.trans_word_to_sound(\"安全驾驶，请注视前方\",'tst_sound.mp3')\n",
    "\n",
    "#                     os.system('mplayer ' + 'tst_sound.mp3')      \n",
    "\n",
    "                    \n",
    "#                 #吸烟概率大于0.5（概率大于50%）\n",
    "#                 elif driver_attr['smoke']['score'] > 0.5:\n",
    "\n",
    "#                     word_2_sound.trans_word_to_sound(\"为了您和他人的健康，请不要在车内吸烟\",'tst_sound.mp3')\n",
    "\n",
    "#                     os.system('mplayer ' + 'tst_sound.mp3')\n",
    "\n",
    "                \n",
    "#                 #其他情况，可以自行添加\n",
    "#                 else:\n",
    "\n",
    "#                     pass\n",
    "\n",
    "            \n",
    "#             #等待10秒后，在进行图像采集\n",
    "#             time.sleep(10)\n",
    "\n",
    "            \n",
    "#     except KeyboardInterrupt:\n",
    "\n",
    "#         os.remove('tst_sound.mp3')\n",
    "\n",
    "#         os.remove('read_word.jpg')\n",
    "\n",
    "#         print(task_name + \"任务被终止\")\n",
    "        \n",
    "# if __name__ == \"__main__\":\n",
    "\n",
    "#     try:\n",
    "\n",
    "        \n",
    "#         mydict=multiprocessing.Manager().dict()\n",
    "\n",
    "        \n",
    "#         #　定义传递图像队列和传递图像处理结果队列\n",
    "#         q_frame = Queue()\n",
    "\n",
    "        \n",
    "#         #　采集摄像头进程、处理图片进程、播报语音信息\n",
    "#         get_camera_frame = Process(target=camera_frame_func, args=(\"获取摄像头图像\", q_frame, mydict))\n",
    "\n",
    "#         proc_frame       = Process(target=proc_frame_func, args=(\"处理图像\", q_frame, mydict))\n",
    "\n",
    "        \n",
    "#         # 启动任务\n",
    "#         get_camera_frame.start()\n",
    "\n",
    "#         proc_frame.start()\n",
    "\n",
    "\n",
    "#     except KeyboardInterrupt:\n",
    "\n",
    "#         print(\"任务被终止了\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 课后练习\n",
    "\n",
    "1、我们继续完善小派的安全驾驶，我们可以添加：发现驾驶员没有系安全带、未带口罩等不安全驾驶行为时，将照片保存下来，并在照片上记录上时间。\n",
    "\n",
    "2、我们还可以增加行车记录功能，将驾驶全过程进行视频记录。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
